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(54) Technique for embedding a code in an audio signal and for detecting the embedded code 

(57) A code is embedded into an audio product so 
as to be processed therewith for recording and/or 
broadcast and yet be reliably detected while remaining 
inaudible to human perception. The code is represented 
by symbols formed from an impulse function having its 
energy within a specified frequency range. The audio 
product is analyzed to find segments which can mask 
the code based on tonality and a minimum signal 
energy. When the audio product with an embedded 
code is detected, decoding thereof involves finding can- 
didate code signals which are checked against preset 
criteria. In particular, each symbol is made of at least 
two impulse functions with a preset spacing therebe- 
tween. 
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Description 

[0001] The invention is directed to an improved technique for coding an audio signal and, in particular, to embedding 
a code into an audio signal so that a decoder can detect the code reliably despite signal degradation. 
5 [0002] Audio signals are generated in a variety of ways, such as by radio and television stations, and transmitted in 
various ways, such as by means of airwaves, cable and satellite as well as distributed on magnetic tape and storage 
disc (e.g. optic, magnetic) media. Various benefits are derived from identifying these audio signals which constitute 
"audio products" in the form of programs or commercials, for example. The audio products can be broadcast by radio, 
television or cable stations and/or stored on tape, CD-ROM or other media for replay by the consumer. By being able to 
10 automatically distinguish one audio product from another, it becomes possible to perform a variety of services. For 
example, air time verification is possible to verify for an advertiser that a commercial has actually been broadcast, and 
that it was aired in its entirety, at the proper time and in the locations that were paid for. In addition, performance royalty 
revenues can be more accurately calculated based on the frequency with which a piece of music, say, has been broad- 
cast. For these and other reasons, it is highly desirable to know when a particular audio product has been "performed" 
is in the sense that it has been heard by any member(s) of a listening audience. Furthermore, the listening (or watching) 
audience can be measured by having individual members or individual households equipped with devices capable of 
identifying certain designated audio products, and then processing the resultant data. This can help measure the pop- 
ularity of a program so that its value to advertisers can be assessed. Also, the exposu re of an audience to a commercial 
-ean-be-measured1hisway, and such informatiurrearrbe combinedwitn other data to determine the effectiveness of that 



20 commercial in terms of how well it is remembered and/or the resulting purchases made thereafter. « 
[0003] The automation of this identification by the prior art has involved various techniques for embedding a code in 
the audio product. The resulting signal is reproduced by, say, a speaker of a radio or television set. The embedded code 
is also reproduced by the speaker so that it can be detected by a sensing device for data storage and/or processing to 
yield the desired information. Various types of encoding schemes are known. However, they have proved to be unsat- 

25 isfactory for one or more of the following reasons. If the code is easily removable without permission, then the accuracy 
of the desired measurement will obviously be skewed. Therefore, it is important for the embedded code to be "indelible" 
in the sense that it cannot be removed without seriously (or at least noticeably) damaging the audio product. Also, the 
code must not create any audible deterioration in the quality of the audio product itself, i.e. which can be discerned by 
a human listener. Furthermore, the code must have adequate immunity to noise which occurs during the sending, play- 

30 back and receiving operations of the encoded audio product. For example, an audio product is typically exposed to var- 
ious phase shifts and time shifts in the process of being recorded and/or broadcast. In addition, the audio product may 
be compressed by a bit rate reduction system based on psychoacoustic compression techniques, such as EUREKA 
147, DOLBY AC3, and MPEG2. (The term "psychoacoustic" has to do with the human auditory response to a sound 
stimulus.) The code must withstand such processing while still maintaining its characteristics for enabling it to be relia- 

35 bly recovered by the decoder while remaining inaudible. Meeting all of these requirements has proven to be too tall of 
an order for the prior art. particularly when combined with the need to minimize the complexity of the apparatus and 
method, and to carry out the technique quickly and efficiently. 

[0004] One aspect of the present invention provides an improved technique for identifying an audio product with an 
embedded code. 

40 [0005] Another aspect of the present invention embeds the code indelibly. 

[0006] A further aspect of the present invention embeds the code in such a way that it is not discernible to the listener 

when the audio product is reproduced audibly. 

[0007] One other aspect provides an improved encoding technique. 

[0008] Still another aspect of the present invention recovers the embedded code despite signal degradation. 

45 [0009] Yet another aspect of the present invention recovers the embedded code despite signal compression. , 
[001 0] One other aspect of the present invention provides an improved decoding technique. 
[001 1] Another aspect of the present invention enables adaptive masking of the code within the audio product. 
[001 2] One aspect of the invention provides a method and apparatus for embedding a digital code in a digitized audio 
product by filtering the digitized audio product to a frequency band of interest. A tonality indication is determined for 

so each of a plurality of segments of the filtered audio product which indicates the extent to which power is distributed uni- 
formly for frequencies in at least a portion of the band of interest. At least a portion of the digital code is inserted into a 
particular segment from the plurality of segments only if the tonality indication indicates a relatively uniform power dis- 
tribution in that particular segment. 

[001 3] Another aspect of the present invention is directed to a method and apparatus for embedding a digitized code 
55 in a digitized audio product by filtering the digitized audio product to a frequency band of interest, and providing a coding 
signal derived from a band-limited impulse function with a waveform having its energy confined to and evenly spread 
across at least a portion of the frequency band of interest. The digitized code is derived from the coding signal, and the 
digitized code is inserted into the audio product. 
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[0014] Yet another aspect of the present invention is directed to a method and apparatus for providing a digitized code 
to be embedded in a digitized audio product by providing the digitized code as a series of binary bits, and dividing the 
binary bits into groups, each group having a plurality of bits. Coding signals are provided to represent the bits, respec- 
tively. A symbol is derived from the coding signals for each of the groups, each symbol having a plurality of the coding 

5 signals with a preset spacing therebetween. 

[001 5] One other aspect of the present invention is directed to a method and apparatus for encoding and decoding a 
digitized code embedded in a digitized audio product by deriving the digitized code in a form of start, data and end sym- 
bol types, each symbol representing a plurality of bits, and each bit being associated with a coding signal of given polar- 
ity. The start type of symbol is generated to consist of a plurality of the coding signals all of which have the same 

10 designated polarity. The digitized code is embedded in the digitized audio product. The digitized code embedded in the 
audio product is detected, and the detected digitized code is decoded by determining whether the polarity of the coding 
signals on the start type of symbol is the designated polarity and, if not, inverting the polarity of the coding signals in 
the data and end types of symbols. 

[0016] Still another aspect of the present invention is directed to a method and apparatus for embedding a digitized 

is code in a digitized audio product by identifying segments of the digitized audio product into which the digitized code can 
be embedded based on predetermined criteria. Portions of the digitized code are generated for insertion into the seg- 
ments, respectively The digitized audio product within the identified segments is removed, except for.a predetermined 
small percentage of amplitude, to generate modified segments, and the portions of the digitized code are inserted into 
the modified segments, respectively. 

20 [001 7] A further aspect of the present invention is directed to a method and apparatus for embedding a digitized code 
in a digitized audio product by analyzing the digitized audio product to derive measured values for designated charac- 
teristics thereof. Segments of the digitized audio productare located, based on the derived measured values and a set 
of preselected parameters, into which the digitized code can be inserted so as to be masked. The digitized code is 
inserted into the located segments, and a determination is made whether a degree of masking of the inserted digitized 

25 code meets a predetermined level and, if not, modifying values of at least one of the set of preselected parameters. 
Then, the locating and inserting steps are performed again with the modified values. - 
[0018] A still further aspect of the present invention is directed to a method and apparatus for embedding a digitized 
code in a digitized audio product by dividing the digitized code into preselected portions, and representing the portions 
by a plurality of coding symbols, respectively. The spacing of the coding symbols from each other is determined to be 

30 used for embedding the digitized code within the audio product so that the spacing is greater than a predetermined min- 
imum, and the coding symbols are inserted within the audio product based on Undetermined spacing. 
[001 9] One further aspect of the present invention is directed to a method and apparatus for decoding an audio prod- 
uct into which a code of digitized coding signals has been embedded by obtaining a digitized audio product and com- 
paring the digitized audio product with a template of a coding signal to identify candidate coding signals based on 

35 shape. Pairs of sequential candidate coding signals are compared with each other based on preselected characteristics 
to identify which ones constitute the coding signals, and then reconstructing the code from the coding signals identified 
by the comparing step. 

[0020] Since the present invention can be implemented as a computer program running on a conventional computer, 
the present invention can be embodied as a storage medium containing computer code, and as an electrical signal car- 
40 rying such computer code. Further, the present invention encompasses an audio signal generated with embedded dig- 
ital code as described herein above. 

[0021] Embodiments of the present invention will now be described with reference to the accompanying drawings, in 
which: 

45 Figure 1 is a general flow chart for the encoder of one embodiment of the invention. 

Figure 2 is a flow chart for the Analyzer function. 

Figure 3 is a flow chart of the Locator function. 

Figure 4 is a flow chart of the Inserter function. 

Figure 5 is a graph of the 2 ping symbol frame. 
so Figure 6 is a graph of the 3 ping symbol frame. 

Figure 7 is a low pass filter impulse response for a filter such as is used for the Analyzer. 

Figure 8 is a low pass filter frequency response for the filter of Fig. 7. 

Figure 9 is a bandpass filter frequency response for a filter such as is used for the Analyzer. 

Figure 10 is a bandpass filter impulse response for the filter of Fig. 9. 
55 Figure 1 1 is a band pass filter frequency response for a filter such as is used for the Inserter. 

Figure 12 is a band pass filter impulse response for the filter of Fig. 1 1 . 

Figure 13 is a ping waveform spectral content. 

Figure 14 is a ping waveform, time domain. 
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Figure 1 5 is a general flow chart for the decoder of one embodiment of the invention. 
Figure 16 is a flow chart for the filtering and normalization functions of the decoder. 
Figure 1 7 is a flow chart for the ping identification function. 
Figure 1 8 is a flow chart for the symbol identification function. 

5 

ENCODER 

[0022] An embodiment of the invention includes an encoding technique for embedding alphanumeric codes in the 
sound waveform of a sampled audio product. This requires a modification of the sound waveform. However, the encod- 

10 ing technique is designed to render the modification inaudible in the sense that it cannot be discerned by a person with 
normal hearing. The encoding technique modifies only short portions of the sound waveform (the content of these mod- 
ified portions is called a "symbor which is used to form a code) and has no effect on the remaining waveform. Inaudi- 
bility of the symbols is achieved by careful selection of the symbol locations and the manner with which the code is 
inserted. The coding technique uses only a specific frequency band within the audio frequency range which has been 

75 selected to allow the embedded information to be recovered by a decoder (described below), even if the signal has 
passed through a low-quality transmission channel. The technique is also reasonably tolerant of frequency variations, 
such as produced by fluctuations in record/playback tape speeds. 



OVERVIEW ~ 

20 <f 

[0023] A general description of the encoder of an embodiment of this invention is provided in relation to Fig. 1 . The 
original sound waveform of the audio product is received as a serial stream of data and is stored into memory in the 
form of a digital audio Input File 1 , e.g. in WAVE format (Multimedia programming interface and data specification No. 

V i .V il \ji 1 1 lUivi ai iu i vuisi vov iky, 1*1 nv< i vcti i win i^i iiivkv utwiCv> ■ i iu m& • v wv« ■• vi • ■ t* wiyuui wwmi <u» i m ■ uuuiu v»i^ 

25 nal or in a well known way by digitally sampling an analog audio signal. The sampling frequency is preferably either 
44,100 samples/sec. or 48,000 samples/sec. The sampling frequency of 44,100 samples/sec. is the standard adopted 
for professional CD recordings and also by some radio stations. The sampling frequency of 48,000 samples/sec. is 
used by most radio stations and also is the standard adopted for digital TV. The encoded audio Output File 1 3 produced 
by the encoder is also stored in the form of a WAVE file. 

30 . [0024] The operation of the encoder is controlled by a number of selectable parameters 8. It may be necessary to 
change these parameters on occasion, for certain types of audio products. For example, audio products produced for 
the motion picture industry to be shown in theatres have a high dynamic range (e.g. 90 db) and high peak to average 
signal ratio, whereas the audio products produced for broadcast transmission have a reduced dynamic range (e.g. 40 
db) and a low peak to average signal ratio. Each type requires parameter selection designed for optimal symbol inser- 

35 tion rates. 

[0025] The encoder is preferably implemented to perform three separate functions which are called herein the Ana- 
lyzer, Locator and Inserter functions. Referring to Fig. 1, the Analyzer 3 analyzes the Input File 1 and produces an inter- 
mediate file called Analysis File 5. This process of analysis is not affected by the selectable parameters 8. The Locator 
7 reads the Analysis File 5 and decides exactly where and how the symbols should be inserted, so that they are psy- 
40 choacoustically well masked by the audio product. This process is affected by receiving certain ones of parameters 8. 
A list of symbol description records is written to a file 9, called an Insertion File. The Inserter 1 1 reads the original Input 
File 1 and the Insertion File 9, and implements the symbol insertions described in the latter. This process is also 
affected by receiving certain ones of parameters 8. The resulting encoded audio is written to a WAVE format Output File 
13. 

45 [0026] In situations where the encoding needs to be tried repeatedly, with different values of the parameters, (sych 
as for the adaptive masking described below or for verifying quality performance) the Analyzer 3 only needs to be run 
once, since its operation is not affected by the parameters 8. Since the Locator and the Inserter functions run faster than 
the Analyzer, this design allows such iterative insertion steps to be done efficiently and quickly. 
[0027] The string of data signals constituting the code to be embedded is preferably a group of hexadecimal digits, 

so each hexadecimal digit representing four bits, or two insertion symbols. The number of bits in the string must always be 
even. If the number is not a multiple of four, the last two bits are separated by a decimal point and expressed as a digit 
in the range 0..3. The order of symbol insertion is left to right in the string, and most significant to least significant within 
each digit. For instance, the string 75E8.2 corresponds to an 18-bit sequence as shown below in Table 1 : 

55 
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Table 1 



5 



7 


5 


E 


8 


2 


01 


11 


01 


01 


11 


10 


10 


00 


10 



■ [0028] A single V character can be appended to any string up to 64 bits long. This causes 14 extra bits to be internally 
calculated and appended to the code. These bits are check digits generated by a (n+7 ; n,5) BCH code in the GF(2 2 ) 
10 (quaternary) field, where n is the number of supplied bit-pairs, up to 32. This code can correct up to 2 erroneous sym- 
bols and detect all 3-error cases. The generation of the check code involves effectively zero-rpadding the supplied bit 
string on the left to a full 64 bits, performing the check digit calculation on those bits, and then discarding the padding 
bits. In other words, the 14 check digits generated for the string "31E4+" are identical to those generated for 
"0000000000003 1E4+". 

is [0029] The Inserter receives data from both the Input File 1 and the Insertion File 9 in order to generate Output File 1 3. 
[0030] The following table is a list of the parameters 8, supplied to run the Locator function 7 and Inserter function 1 1 . 
The parameters are listed along with their default values. The meaning and utilization of these parameters is explained 
below. 



Table 2 « 



Name 


Units 


Default 


Description 


MinEnv 


Signal 


1200 


) 


Control the Masking 


MaxRalio 


None 


300 


) 




MinRatio 


None 


0 


) 




MaxSide Ratio 


None 


25 


) 




MinPingSpacing 


Seconds 


0.1 


) 


Control the Spacing 


PingDither 


Seconds 


0.02S 


) 




PingSpacingAlpha 


None 


1.3 


) 




PingSpacingBcta 


None 


2.4 


) 




BandRemoveFac 


None 


0.98 


) 


Control the scaling and 


PingGain 


None 


0.4 


) 


insertion of a ping. 


PingGainMode 


None 


1 


) 





40 [0031] The encoder confines its operations to a frequency band of approximately 1000 Hz to 5000 Hz because it has 
the predominant amount of audio program energy content and is the least deteriorated by sending, playback and 
receiving. Thus, it is more robust in terms of resistance to distortion by various effects caused by the processing which 
the audio product must undergo, such as compression. Signal components outside this band are not affected by the 
encoding process, and are largely ignored by the signal analysis process of the Analyzer 3. This frequency band will be 

45 referred to as the "band of interest". a 

[0032] When a symbol is inserted, the sound waveform in the original audio product, in the band of interest only, is 
removed for a time period allotted for insertion of the symbol, called a symbol insertion period (used interchangeably 
herein with "segment"). The removal is not complete in the sense that some of the waveform of the original audio prod- 
uct is retained throughout this period, and the removal is implemented with a soft ramp at the beginning and end of the 

so symbol insertion period. The removed audio is replaced by two, or sometimes three band-limited impulse functions, 
spaced at preselected intervals from each other. Data coding is performed by setting the polarity of these impulse func- 
tions. Each impulse function, called a 'ping', is an ideal mathematical impulse function to which a steep bandpass filter 
has been applied. The energy of the resulting waveform is confined to. and evenly spread across, the range of 1 485 Hz 
to 3980 Hz. 

55 [0033] Heard alone, these 'ping' symbols sound like a 'click* or 'pop'. The task of the Locator 7 is to find segments of 
the audio product in Input File 1 in the band of interest which are sufficiently spectrally rich (as defined below). When a 
ping waveform is added to such a segment, it sufficiently resembles the energy levels that were removed so that the 
modification is therefore difficult for a person to discern audibly. The magnitude of the inserted symbol is scaled to the 
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RMS power of the in-band envelope signal in the symbol insertion period (the term "in-band" refers to the frequency 
range for the output signal of low pass filter step 20 in Fig. 2). This is done for two reasons: firstly, it helps provide a 
consistent 'replacement* power of the removed audio, to reduce the audibility. Secondly, the decoder (as described 
below) uses a normalization (or automatic gain) process which scales the signal it receives to provide a uniform RMS 
5 power level in the band of interest. By scaling the ping waveforms to the surrounding RMS, the encoder helps ensure 
that the ping waveforms appear in the decoder with a predictable amplitude, namely scaled to the amplitude of the sur- 
rounding audio. The ping symbols are sufficiently structured and unusual that they can be identified by the decoder with 
a high degree of accuracy within the surrounding audio, even after allowing for some distortion in the course of trans- 
mission. 

10 [0034] The signal processing generally follows a model wherein a digitized signal is operated on by a signal process- 
ing function, the result being another digitized signal. This signal can then be passed to one or more further processing 
blocks. Generally speaking, a "signal" is a stream of 32-bit floating point numbers, or samples, representing the voltage 
on a hypothetical physical signal at equally spaced points in time. Due to the nature of the processing being performed, 
it is necessary to relate each output point to a specific time point in the Input File 1. In other words, all propagation 
* is delays through the signal paths need to be tracked very carefully. This is complicated by the fact that different signals 
have different sampling periods, and different signal processing elements have different delay characteristics. All sam- 
pling periods and delays are expressed in integral multiples of the "fundamental tick", or ftick. The ftick period is about 
88.6 nsec. The exact value of this time period depends on the input file sample rate, as explained in detail below. 



20 ANALYZER , 

[0035] The Analyzer 3 generates two measurements of the input signal, namely the in-band envelope and the Tonality 
Ratio. The in-band envelope is a moving-average measure of the RMS power in the in-band signal. The Tonality Ratio 
is a figure indicating whether the in-band signal appears io be speciraiiy rich, i.e. having relatively uniform power 
25 throughout the band, or whether it is tonal, i.e. having a mix of high and low powers. Tonal signals do not mask clicks 
and pops well. 

[0036] The envelope measurement is done via a series of signal processing stages as described below. The process 
is designed to produce a signal averaging envelope in a way which matches what wili be done in the decoder for the 
same audio product, as explained in detail below. 

30 [0037] As shown in Fig. 2, step 20 converts the signal from its sampling rate of 44100 or 48000 samples/sec. to a 
sampling rate close to 11 025 samples/sec. All later processing is based on the latter signal. Step 20 also applies a low 
pass filter which removes all frequencies above about 4950 Hz so that they have no effect on the encoding process. 
[0038] Proceeding to signal path 22 of Fig. 2, the 1 1025 samples/sec. signal is passed through a bandpass filter 24 
with a frequency range of 980 Hz to 4850 Hz. Then, a power measurement process 26 reduces the sampling rate by a 

35 decimating factor of 8. Step 26 takes in eight samples from the bandpass signal outputted by step 24 and sums the 
squares of each sample, and divides the result by 8. One result is outputted for each eight input samples. An 8 times 
decimating and low-pass filter 28 is then applied for smoothing the output of step 26. Filter 28 is a 31 tap filter with the 
fixed coefficients set forth in Table 3. 

40 

Table 3 



45 



50 



55 



OFFSET 


VALUE 


OFFSET 


VALUE 


0 


0.00364725799282 


16 


0.0610569733715 


1 


0.0071953136762 


17 


0.0596727757011 


2 


0.0112656333407 


18 


0.0574157779489 


3 


0.0157909231222 


19 


0.0543587323779 


4 


0.0206811320698 


20 


0.0505991154649 


5 


0.0258259243044 


21 


0.0462550106913 


6 


0.0310981949734 


22 


0.0414601700119 


7 


0.0363584938162 


23 


0.0363584938162 


8 


0.0414601700119 


24 


0.0310981949734 


9 


0.0462550106913 


25 


0.0258259243044 
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Table 3 (continued) 



5 



w 



OFFSET 


VALUE 


OFFSET 


VALUE 


10 


0.0505991154649 


26 


0.0206811320698 


11 


0.0543587323779 


27 


0.0157909231222 


12 


0.0574157779489 


28 


0.0112656333407 


13 


0.0596727757011 


29 


0.0071953136762 


14 


0.0610569733715 . 


30 


0.00364725799282 


15 


0.0615234375 







[0039] The filtered result from the sum of the squares outputted by steps 26 and 28 is applied to step 29 which cal- 
is culates the square root and inverts the signal. Step 30 re-inverts the signal generated by step 29 to produce the RMS 
in-band envelope signal 32. Signal 32 provides two envelope points for each of a plurality of periods of predesignated 
duration (e.g. symbol insertion periods), namely the RMS in-band envelope at the beginning and at the centre of such 
period. 

[0040] The resulting signal 32 tracks the RMS level of the in-band signal, with a slow response time to reflect the 
20 response of the human auditory system to perceived loudness variations. The Locator 7 will not insert symbol^unless 
this envelope level is above a certain adjustable threshold. The principle is that quiet material is more likely to be cor- 
rupted by noise in the transmission channel and, therefore, does not provide a safe insertion point for a symbol. 
[0041 ] • Proceeding to signal path 34 in 1 Fig. 2, the Tonality Ratio is measured via a Fourier-Transform process 36 which 
operates on the signal produced by the step 20. A 1024 samples FFT (Fast Fourier Transform) is used at spacings of 
25 128 samples, on the time domain 1 1025 samples/sec. data, which provides a frequency resolution of about ,1 1 Hz in 
the frequency domain. The resultant FFT signal is divided into 512 frequency bins. However, of these only the 320 cen- 
tral frequency bins are used, and these 320 bins are divided into 16 groups of 20 bins each, per step 38. In step 40, the 
total power is calculated within each group. Also, within each group, a measure of the range of power values present in 
the group is made. This measure is the ratio of the second-highest bin power divided by the second lowest. A weighted 
30 sum of these measures is made across all 16 groups, per step 42, using the individual group powers as weights. The 
result is the Tonality Ratio. V 

[0042] The Tonality Ratio tends to be higher for a tonal signal, i.e. which contains a mix of high and low bin power 
values within each group, and lower for spectrally rich signals, i.e. which have relatively uniform bin power across each 
group. Tests have shown that this measure tends to correlate very well with the sound's ability to mask the symbols. 
35 Symbols will be inserted in locations where the Tonality Ratio is lower than a preset adjustable limit (i.e.. in a spectrally 
rich sound waveform). Also, the Tonality Ratios generated by the preceding and following symbol insertion periods from 
the FFT analysis operations are compared against a less restrictive limit (i.e., a higher preset limit) than the one used 
for the present symbol insertion period. This technique ensures that the symbols are well hidden, or masked, in the 
audio product. 

40 [0043] Step 43 determines the total in-band power based on the FFT signal. In particular, as explained below, this 
total power is a summation of the in-group powers. 

[0044] Thus, for Analyzer 3 as shown in Fig. 2, the input is at 48000 or 44100 samples/sec, and the output is a stream 
of records 44, which are stored in the Analysis File 5; Each record contains two envelope points, one Tonality Ratio 
value and one value of the total in-band power from the FFT which are interleaved for a single symbol insertion period. 
45 [0045] Further details regarding the sampling rate will now be provided. The system handles different sample rates 
by sample-rate converting the Input File 1 to a common internal sample rate, which is always very close to 11025 sam- 
ples/sec. The Output File 1 3 is always at the same rate as the Input File 1 , allowing the output to be numerically identical 
to the input, except where symbol insertion is being done. 

[0046] To simplify the handling of different sample rates, the ftick is defined. This is a time period whose exact value 
so depends on the audio sample rate. Given f w the sample rate of the Input File 1 , the value n is found, being the closest 
even integer to 1 1025 • 1 024/f w . 

[0047] The time period 1/nf w is known as the ftick. This will always be close to 88.6 ns. The ftick is used as the meas- 
ure of time within the system. All sampled signals have periods which are multiples of the ftick. For instance, the low 
pass filter signal at the output of step 20 to which the input is sample-rate converted, has a period of 1024 fticks. Its 
55 sample rate will approximate 1 1025 samples/sec. to a few parts per thousand. The table below shows the rate of fticks, 
and the sample rate of the low pass filter for input rates of 44100 and 48000 samples/sec. which is the first signal after 
down-conversion. Most internal signals have rates equal to this, or decimated by a power of two. 
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Table 4 



w 



Input Sample Rate 


44100 


48000 


n 


256 


236 


Fundamental Tick Rate 


11289600 


11328000 


Low pass filter (Step 20) 


11025 


11062.5 



[0048] The Analyzer has seven data members which are intermediate 
described below. 



processing results. These members are 



15 



20 



25 



30 



35 



40 



Table 5 



Member 
Input FiIp 1 


Sample Period (Funda- 
mental ticks) 

236 or 256 


Description 


Step 20 


1024 


-lnput-signal-(48OO0-or-444OO-samples7sec-) - 

Low-pass and sample-rate converted signal 


Step 24 


1024 


Signal after band-pass filter 


Step 26 


8192 


Signal after power measurement 


Steps 28, 29 


65536 


Signal after filtering of power measurement and inversion 


Step 30 


65536 


Signal after final filter and re-inversion 


Step 36 


65536 


Signal generated by FFT analysis 



[0049] Details of the steps in signal path 34 of Fig. 2 will now be provided 

[0050] To run the FFT operation of step 36. an FFT sample size of 1 024 is read from the output of step 20 and a cosine 
(Von Hamm) w.ndow function is applied to the time domain data. An input pointer is moved by 1 28 samples after each 
foSg'Son 0 ^ 3 SUbStantia ' ° Verlap b6tween successive FFT operations. Each FFT operation performs the 

1 . M real numbers are presented as a waveform to be transformed. M (i.e. 1024) is the FFT size used 

2. The time-domain window is applied to the waveform, the FFT is performed, and then the power for each of fre- 
quency bins from 0 to (M/2)-1 is calculated as the square of the magnitude of the complex FFT result. The resulting 
complex spectral image is simplified by retaining the square of the magnitude of each frequency bin and discarding 
the phase information. a 

3. M/2 real numbers are then retrieved. For M * 1024, these are the power values for frequency bins 0 to 51 1 . 

[0051 ] This FFT operation can be described mathematically as 



45 



so 



2nj -~} I*-*" 



- £ w(t)x t e " £ w(t)x t e 



\t - o 



o. .f -1 

2 



55 



... where x t is the input waveform, M is the FFT size, w(t) is the window function, and F, is the magnitude-squared 
SLilig) 8 ' S P6rf0rmed S0, f ° r instance ' the output in bin zero is the ***** of the sum <> f M inputs (after 
[0052] each FFT result is. thus, obtained in the form of a power spectrum: A set of frequency bins, numbered from 0 
to 51 , containing the square of the magnitude of the complex FFT result for the corresponding frequency. The n-th bin 
contains information for a frequency of 
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n Wi Hz = n ■ < io77Hz ) 



JO 



[0053] In accordance with step 38. the subset of this power spectrum corresponding to the band of interest is divided 
into groups. There are 16 groups, each having 20 frequency bins. The lowest group starts at frequency bin 96. Bins 0- 
95, as well as 437-511, are discarded. So these 16 groups cover the frequency range from about 1034 Hz to about 4470 
Hz, each group covering about 21 5 Hz. 

[0054] Within each group, the following three determinations are made in accordance with step 40: 

• The sum of the power values in each of the 20 bins (total group power) 

• The second smallest power value in the 20 bins 

• The second largest power value in the 20 bins. 



is [0055] Of course, the second smallest and second largest power values could be replaced by values of other bins, 
the main object being to discard the highest and lowest values as possible statistical aberrations. For each group, the 
second largest bin power is divided by the second smallest, and then the following non-linear clipping operation is per- 
formed: If the result exceeds 2000, then half the amount by which it exceeds 2000 is deducted. If the result of this oper- 
ation exceeds 4000, then it is set to 4000. This provides a unitless measure of the power skew in the group. If the 

20 second smallest bin power is very small (as defined below in the equation for Tj ), the power skew for the group i£ simply 
set to 4000 to avoid division by a very small number. 

[0056] A weighted average is taken of the square roots of these ratios. The weighting value for each group is the total 
group power. The results provided by step 40 are squared and become the Tonality Ratio result for the whole FFT oper- 
ation, as per step 42. It will always be at least one and no more than 4000. If P\ is the total power obtained by summing 
25 the power of the bins in group i. and Bj and Lj are the second-largest and second-smallest bin powers, the. calculation 
can be summarized as 



30 



IB,!., 



1 ,B ± 



if B i /L i <> 2000 



Lj " 2 <7p*000), or 2TT+1000 if V*i > 2000 and B.L, s 6000 
4000 otherwise 



35 



40 



45 



15 



E 



i - 0 



15 
i - 0 



.. where T is the overall Tonality Ratio. 

[0057] There is also an RMS power calculation for the total in-band power which is derived from the FFT signal. The 
so calculation of such total in-band power is as below: 



55 



1024 ^0 1 
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[0058] The division by 1024 corrects for power gain in the unsealed FFT operation, and the square root converts from 
a power measure to an RMS measure. The factor corrects for attenuation due to the cosine window. If the input is a sine 
wave of amplitude A, with a frequency in the band of interest, the resulting P' will be close to 



10 



15 



which is the correct RMS measure for a sine wave. 

[0059] A data record 44 corresponds to information about the sound waveform of the audio product for a symbol inser- 
tion period, or about 11.6 msec of real-time for a 44,100 samples/sec. rate. 
[0060] There are four data values in the record: 

1 . The ia-band envelope level (from step 32) at the beginning of the 1 1 .6 msec period. This is in signal units and is 
approximately equal to the R.M.S. value of the in-band signal. It is rounded and stored as an unsigned 16-bit inte- 

ger. z?.:. 

2 ' ^-band envelope level (from step 32) atthecentreof the 11.6 msecperiod. Same units and scaling are used 
as for the first envelope point. 



so 



-3^rre-TonalirRatto-(»rom step-^rfoTafrFh ' calculation centered about the begi nning of the 1 1 .6 msec period 
This is a umtless number in the range 1 -4000; it is scaled by 2 3 and stored as a 16-bit unsigned number , ' 
4 The total in-band power (from step 43) based on the FFT signal for a time period centered about the beginning 
of the 1 1 .6 msec period. This is in signal units and is approximately equal to the R.M.S. value of the in-band sianal 
It is rounded and stored as an unsigned 1 6-bit integer. 

25 Mathematical Background 

[0061] The following description supplies the mathematical background underlying some processes used in the Ana- 
lyzer 3. General methods are given for 

30 • Sample Rate conversion, by a rational ratio less than unity; 

• Construction of a low-pass, linear phase FIR filter from arbitrary parameters; v 

• Construction of a band-pass, linear phase FIR filter from arbitrary parameters; 

[0062] A down-sampling process of step 20 converts the Input File signal to the low pass filter signal, simultaneously 
35 reducing the sample rate by a rational factor and removing higher frequencies which cannot be represented in the lower 
sample-rate signal. The Input File 1 is resampled in such a way as to reduce the sample rate by a factor of K/V K is the 
factor by which the original sample rate is multiplied. V is the decimation factor. It is assumed that K<V, but the same 
approach works for increasing the sample rate. The ratio must be in lowest terms; i.e. K and V have no divisors in com- 

40 [0063] Conceptually, the signal is processed through the following individual steps: 

1 . The input signal is called Xj. 

2. The rate of y, is increased by a factor of K, giving the signal This is done by inserting K-1 zero samples after 
each sample: 



45 



x± i is a multiple of K 

K 

otherwise 



50 



55 



3. A low-pass FIR (Finite Impulse Response) filter with impulse response F k of length L • K is applied to this sianal 
giving the signal y\: * 
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LK-l 
k - 0 



4. The result is decimated by discarding all samples except those whose indices are a multiple of V, leaving the sig- 
10 nal yj: 

yj = /iv 

[0064] These steps produce a rate-converted signal which is as close as possible to the mathematical ideal. The filter 
is used in step 3 is ideally a brick-wall' low-pass which removes all frequencies too high to be represented at the new 
sample rate. Since an ideal filter has an infinite impulse response, the FIR can only approximate this ideal. A certain 
amount of signal aliasing will occur in step 4, which also arises because of the non-ideal FIR filter. Step 2 reduces the 
amplitude of the signal in the band of interest. This is compensated in step 3 by arranging the filter F to have a passband 
gain numerically equal to K. 

20 [0065] The process described just above can be simplified by capitalizing on the fact that many of the x'j samples are 
zero and many of the y'j samples are discarded. 

[0066] The FIR filter F k used in step 3 can be rearranged into matrix form: 



25 Gn = F, 



j = 0 ... L - 1 



k.j ~ C k + Kj lc - 0 . . . JC - 1 



so 

30 



35 



k - 0 j - 0 



x'j is zero except when i is a multiple of K, i.e. when (i-k-Kj)MOD K = 0 in the equation above. An equivalent condition 
is k = i MOD K . Thus, the outer summation can be eliminated, using a single value for the index variable k: 



40 



45 

where k = i MOD K . 
[0067] With this definition of k, we have 



so i - k = i - (i MOD K) = 



- HI" 



So, 

55 



11 



where k = i MODK . 
From the definition of x\, 



10 



15 



and, from the definition of Y;; 
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L - 1 



L - 1 



Vi ' ■ i ?o G(iM, - /, WH 



L - 1 



20 



3 ~ 0 



VMODk) , j 



25 



[Q06S] This last equation performs the whole convetsion in a singie siep. oy substituting 



and 



kj = iV MOD K 



30 



35 the last equation can be rewritten as 



40 



L - 1 
3 - 0 



[0069] Each summation operation is equivalent to applying a size L Fl R filter to the input signal x starting at offset %. 
The coefficients of the FIR -its impulse response-are the row of G selected by k-. Since the overall filter F has a gain 
45 of K, each row of G has a passband gain approximately equal to unity. 

[0070] Assuming that the FIR is a symmetrical, zero phase low pass design, the delay incurred from signal x'j to y • is 
(LK-1 )/2 samples. The other conversions do not introduce delay, but they change the sample rates. The overall delay is 



so 



4 



input samples, or 



55 



LK-1 
2V 
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output samples. 

[0071] The rate conversion done on the input sample rate is a reduction by a factor of n/1024, where n is the number 
of fticks in the period of the input audio file. Thus, the conversion technique can be applied with 

"■5 

where C is the greatest common factor of 1024 and n. the delay, expressed in output samples is thus 

15 L« -1 



10 



2 M2i 



20 



or 



25 



30 



35 



40 



nL-C . 
2* 1024* 



[0072] In terms of fticks. the delay of the conversion process is (nL - c)/2 fticks. Since n is even, C will also be even, 
so this will be a whole number. 

[0073] A generalized low-pass FIR filter with linear phase delay can be constructed using the general equation 

Fj = 2^sind2^ (i - i 0 )\w{i,L) 

where 

L-1 • , x sin(rcx) 
0 2 nx 

L is the length of the filter, f is the sampling frequency, fc is the cutoff frequency, and w is a window function which 
applies over the range [OK L - 1]. The Kaiser-Bessel window function 



45 



w k (i,L) = 



i - i 21 : 1 - 1) 2 



Io(P) 



so 



is useful because the beta value can be adjusted to control the trade off between sharp cutoff and good out-of-band 
rejection for a given filter length. IO(x) is a modified Bessel function. 
[0074] All frequencies are delayed V£(L - 1 ) samples by such a filter. 
55 [0075] It should be noted that sinc(x) needs to be evaluated as 1 , or, more accurately, as 1 -1/6(rcx) 2 ( for values of x 
close to zero. 

[0076] By multiplying the low-pass filter with the centre frequency of the passband, a generalized band pass FIR filter 
can be realized as 
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~S 



inc|-^(i - i 0 ) jcos^Tt^ (i - i 0 )jw(i,L) 



where 



, 0= if,sinc (x) ^ 

as before; L is the length of the filter, fs is the sampling frequency, fw is the width of the pass band, fc is the centre fre- 
15 quency of the pass band, and w is a window function. 

LOCATOR 



[G077] As-showrrirrFigr37the Locator analyses records~44 from the Analysis File 5. As stated above, each record 44 

20 contains information for about 11.6 msec of the audio product. The Locator reads every one of these records? in 
sequence. As each record is read, a decision is made as to whether to insert a symbol at that point. The Locator is 
divided into three separate processes: 

• The Spacing process guarantees a minimum spacing between consecutive symbols, and prevents symbols from 
25 being inserted at regular intervals. This process does not examine the Input File characteristics, 

• The Masking Test process examines the signal characteristics at each possible insertion point, as determined by 
the spacing process, to determine whether a symbol inserted at that point would be masked well. This process 
does not consider the spacing between insertions. 

• The Code Sequencing process is responsible for sequencing the binary codes represented by the symbols. When 
30 a decision is made to insert a symbol, the Code Sequencing process determines the type of symbol to be inserted, 

as explained below, and the binary data it is to carry. v 

[0078] The spacing determination is made first to identify the minimum spacing for locating the next symbol. The 
masking test is not performed until a record 44 is reached for which this minimum spacing requirement is met. 

35 

Spacing Process 

[0079] The ping spacing process of step 60 (Fig. 3) is controlled by the following four parameters: 

40 • MinPingSpacing (also referred to as "MPS"), expressed in seconds; 

• PingSpacingAlpha and PingSpacingBeta, unitless ratios; and 

• PingDither, expressed in seconds. 

[0080] MinPingSpacing is the minimum time between symbol insertions. The other parameters combine to increase 
45 the minimum spacing in various ways. The earliest time the i-th symbol can be inserted is given by minimum spacing in 
various ways. The earliest time the i-th symbol can be inserted is given by 



so 



t i * Max 



^i-i * MinPingSpacing 
t^ 2 + MinPingSpacing * (1 + a) 
^.3 + MinPingSpacing • (1 + a + P) 



+ PingDi ther - r i 



55 



14 

BNSDOCtD: <EP 0913952A2_I_> 



EP 0 913 952 A2 



where tj is the insertion time of the i-th symbol 
a = MinPingspacing Alpha 
P = MinPingspacingBeta 

r s is a random value uniformly distributed in [0,1]. 

5 

Values of tj where i<0, are assumed to be zero. The rj random variable is generated each time a new symbol is inserted, 
and its effect is to ensure that the interval between symbols is random. 

[0081] Typical audio products contain a continuous sequence of segments which are suitable for symbol insertion. 
This Spacing Process prevents rapid, regularly spaced insertion of symbols, since such are detectable to the ear 
10 because of their regularity. Selection of a and p is made empirically by listening to their effect on audibility. 

[0082] The following table has been derived for illustrative purposes only. The arbitrarily chosen parameter values are: 
MPS=3. a=2 and p=1. For the sake of simplicity, fj is not taken into account in this example. 



Table 6 



Symbol 


tM+MPS 


tj_2+MPS(1+a) 


ti. 3 +MPS(1+a+P) 


Min tj based on Spac- 
ing Test 


Actual tj based on Mask- 
ing Test 


1 










0 


2 


0+3=3 






3 




3 


3+3=6 


0+3{3)=9 




9 


12 


4 


12+3=15 


3+3(3)=12 


0+3(4)=12 


15 


16 


5 


16+3=19 


12+3(3)=21 


3+3(4)= 15 


21 


23 


6 


23+3=26 


16+3(3)=25 


12+3(4)=24 


26 


26- 


7 


26+3=29 


23+3(3)=32 


16+3(4)=28 


32 


32 


8 


32+3=35 


26+3(3)=35 


23+3(4)=35 


.35 


35 


9 


35+3=38 


32+3(3)=41 


26+3(4)=38 


41 

• .... k 


46 


10 


46+3=49 


35+3(3)=44 


32+3(4)=44 


49 


50 



[0083] Often used values for these parameters are: MPS = 0.1 sees, a = 1 .3 sees, p = 2.4 sees. 
35 [0084] If step 62 indicates that a particular record meets the minimum spacing requirement, then the masking test is 
performed on that record and all subsequent records until a symbol is inserted. Then the ping spacing process of step 
60 is performed again in order for a new minimum spacing to be calculated and applied. 

Masking Test Process 

40 

[0085] The Masking Test process 64 compares the signal characteristics for records obtained by step 62 from the 
Analysis File 5 against limits set by the encoder parameters 8. This process is controlled by the following parameters: 

• MinRatio and-MaxRatio control the acceptable limits of the Tonality Ratio. 

45 • MaxSideRatio sets the acceptable maximum for the Tonality Ratio in adjacent measurements. The Locator has one 
frame of look-ahead', so it must read the next record before deciding whether to insert a symbol at the time corre- 
sponding to the present record. 

• MinEnv sets the minimum acceptable value of the signal envelope measurement. 

so [0086] MinEnv is in signal units; the others are unitless. 

[0087] If the symbol Vj represents the Tonality Ratio as retrieved by step 64 from the i-th record in the Analysis File 5, 
and 6j is the in-band envelope estimate for the same point in time, then a symbol can be inserted at the corresponding 
time point if 

55 
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v i z MinRatio 

v £ <l MaxRatio 
V s _ x z MaxRatio • MaxSideRatio 
V in * MaxRatio • MaxSideRatio 
e x * MinEnv 



w 



axe all true. 



SidtT ^ above ' each recorcf 44 ha **° in " band envelope ,eve,s ' Bo,h ,eveis must meet ,he 

[0089] The purpose of MinEnv is to prevent the encoder from inserting symbols into portions of the airiin nm* m 
-wh.Gr^are^o-quieHhaHhey.may-be-ovewhelmed-brnoiseirrtrr e a « u fa i i liZ L iO f 7. channel P - 



EL nml^f r ^"l" ensure that ,he P ro P osed *»««on site, and the surging area of the signal are 6 uf- 
ficiently non-tonal or have sufficient psychoacoustics to mask the symbol well 9 ' 

K S 5 r 6556 " t S 3 ratl '° rel3tiVe 10 MaXRa,i ° ( : ' MR ")- N0rma »y MSFVMR is >1 

ESLI'i'Sri! J^?!' 0 .f PPr0acheS 1 : the . mor _ e *" (in the sense of having substentia.lv uni- 

-I ^ T '" ; l " e aua, ° proaua M,nRa,l ° establishes the noise floor for symbol insertion if «. 
segment ,s extremely no,sy and is then subjected to noise reduction or removal systems symbo s TserteSS 1 
no.sy segment could be removed and lost through the same process By defininc , ftSiSfc far Ton^ ? * 

SET 1 are acted upon as w audi0 and therefore •rt-^S^b^^ 1 ^'' 11 rt can be 

" SSSfSJS* ,S 1 ' iPSerti0n ,6Vel iS Pin9Ga,n the ^ env6,0pe level at ,he inser «°" «™ (i.e. out- 

• t^^^^Z^. times the total in - band power as e * imated * Fouri - 

Sfonhln^ 0131 l ?? nd ^ 88 eStimated by the FFT at ,he time of "*«tion with step 43, is a more accurate meas 
10096] ^^ot^m^^em^^lyb^^^a^o^^m,^^,^^ ' 

Code Seq uencing 

W eZ t£ S 52T C0d6 ' ^ - nUmb6r ° f Di,S " ,en9th - ^ — *° b ^ 

reference in the receiver, ind.cating whether the channel has inverted the signal esiaoiisnes a 

• n-1 data symbols, which encode 2 bits each; 

• An end symbol, which encodes the last 2 bite of the codeword and also serves to mark the end of the codeword 



25 



30 



35 



40 



45 
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with the addition of a conditional ping. 

[0099] The start and end symbols are each coded as 3-ping symbols, while the data symbol is a 2-ping symbol The 
start symbol is coded by the bit pattern *000\ meaning three negative pings. The two pings in a data symbol are coded 
s according to the two bits being encoded. In an end symbol, the first two pings are encoded according to the data, and 
the third ping is the inverse of the second ping of the symbol. 

[0100] The table below shows an example where the 12-bit sequence '0110 1 100 1 1 10' is encoded with seven sym- 
bols: 

10 

Table 7 



20 



Bits 


Coded 


Ping Polarities 


Symbol Type 




000 




Start 


01 


01 


- + 


Data 


10 


10 


+ - 


Data 


11 


11 


+ + 


Data 


00 


00 




Data 


11 


11 


+ + 


- Data 


10 


101 


+ - + 


End 



25 [0101] Thus, in step 70 of Fig. 3, the data string of code sequencing determines the present position reached in the 
code to be embedded during the embedding operation of the encoder, as shown above in Table 1 . If the operation is at 
the beginning of such code, then step 70 will provide the start symbol for insertion. Likewise, if a data bit is to be 
encoded, as per the table presented above, the corresponding code symbol type and code will provided. Step 71 stores 
into Insertion File 9 the symbol type and data code received from step 70 as well as the ping level from step 68. This is 

30 stored in association with a particular segment of, or location in, the sound waveform of the audio product. 

[0102] . When specified conditions for symbol insertion have been met and a symbol is to be inserted, step 72 causes 
the spacing process and code sequencing to be updated. 

I NSERTER 

35 

[0103] The Inserter 1 1 reads the Input File 1 and writes a new WAVE audio file which is identical to the Input File 1 , 
except for short sections which are modified by symbol insertions made according to the information in the Insertion 
File 9. 

[0104] The Insertion File 9 provides the following information for each symbol: 

40 

Insertion time (given in fticks) 

• Insertion Gain (as an R.M.S. level) 

• Symbol type, and data bits to be coded. 

45 [01 05] The Output File 1 3 must always have the same sample rate and format as the Input File 1 ; e.g. if the Input File 
is 1 6-bit stereo at 48000 samples/sec, then the Output File has the same characteristics. 

[0106] A symbol is inserted into the original audio product by the two processes, Band Removal and Ping Addition, 
described in detail below. The nominal insertion time is the midpoint of the area in which the insertion is performed. 

so Band Removal 

[0107] As shown in Fig. 4, a bandpass signal is derived by applying a bandpass FIR filter to the original signal from 
Input File 1 to remove frequencies outside the range 1000-4800 Hz approximately. This is done per step 80 by using a 
symmetrical, effectively non-causal filter with zero phase response. This filter requires (N-1)/2 samples before and after 
55 the modified section of the signal where N is the filter length, which is odd. 

[01 08] In step 84, a modified bandpass signal derived by step 82 (as explained below) is subtracted from the original 
signal over a period of 7T or 10T. Each T is 256 samples at 44100 samples/sec, or about 1.451 msec. As shown in 
Figs. 5 and 6, a 2-ping symbol occupies 5T and a 3-ping symbol occupies 8T. The duration of signal represented by a 
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record 44 is 8T, or 11 .6 msec. 

[0109] In step 82, first and last sections of the bandpass signal, of length T" each, are tapered with a raised cosine 
window. The windowing operation ensures that no transients are introduced into the signal. Thus, as shown in Figs. 5 
and 6. 7T is used for the 2-ping symbol (i.e. for data type) and the 10T is used for the 3-ping symbol (i.e. for start type 
5 and end type). Also, the bandpass signal is multiplied by BandRemoveFac, an empirically set parameter that is one of 
parameters 8, which is just less than 1. This causes the in-band signal to be almost entirely removed in the symbol 
insertion segment. The retention of some contextual component of the audio product has been empirically determined 
to have a psychoacoustic advantage in masking the removal of the audio and the insertion of the symbol. The retained 
components should be in the range of 2% - 5% of the amplitude of the audio product in the symbol insertion segment. 

10 

Ping Addition 

[01 1 0] Once the band-removal is completed, a fixed 'ping' waveform is scaled per step 86, the symbol is derived per 
step 88 and inserted per step 90 in the signal. Two or three pings (depending on the symbol type) are spaced 3T apart 
is from each other and centered in the segment from which the band was removed. 
[01 1 1 ] Scaling of the ping in step 86 is based on two factors: 

1 . Polarity: *0' bits have negative polarity, while *1 * bits have positive polarity. This information is obtained from Inser- 
tion-File-9rThe-startsymbo|-establishes a reference in case the process of transmission and reception inverts the 

20 signal. ^ 

2. Magnitude: This is calculated based on PingGainMode using the total in-band power or the in-band envelope in 
the symbol insertion area, as per step 68 described above, to mask the audibility of the ping while maintaining its 
recoverability. The magnitude is given as an RMS level, averaged across a 3T period centered on the ping, and it 
is obtained from Insertion File 9. 

25 

[0112] Step 88 derives the symbol from the symbol type and data code information stored in Insertion File 9, and 
based on the scaled ping outputted by step 86. The result is inputted to step 90 which proceeds to perform the actual 
insertion, or embedding, of the symbol. 

[01 1 3] Figs. 5 and 6 show a 2-ping symbol and a 3-ping symbol, respectively. The ping waveform is derived from a 
30 template which has a width of 4T. However, it is almost zero outside the 2T centre area, as shown. The ping waveform 
template will be discussed below in further detail. V 

Bandpass Filter 

35 [01 1 4] This filter is used for step 80. It is constructed using the method given above in the Mathematical Background 
section, with parameters as follows: 

• Passband from 950 Hz to 4850 Hz 

• Kaiser Window with parameter of 7 

40 • FIR Length: 7.78 msec. This is converted to a number of samples at the current sample rate. The result is rounded 
to the nearest integer. If the result is even, one is subtracted. 

Cosine Window Template 

45 [01 1 5] The template for the cosine window of step 82 has for its length a number of samples equivalent to T (1 .£51 
msec), regardless of sample rate. Thus, the number of samples varies with the sample rate. The template fades from 0 
to 1 based on the relationship 

[01 1 6] The template is used to fade the beginning and end of the band removal process. 
so 1 

Template ^h 1 -^ Temp + Le 2 ng^ 



55 Ping Waveform 

[0117] This is constructed as a bandpass FIR filter, using the method given above in the Mathematical Background 
section, with parameters as follows: 
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• Passband from 1 485 Hz to 3980 Hz 

• Kaiser Window with parameter of 8.5 

• FIR Length: 5.804 msec. This is converted to a number of samples at the current sample rate. The result is rounded 
to the nearest integer. If the result is even, one is subtracted.. 

5 

[0118] The resulting array of samples is scaled so that it has an R.M.S. power of 1, average over the middle "3T" 
(4.353 msec) portion of the WAVE file. 

Filter Plots 

10 

[0119] Frequency-domain and time-domain response plots for the three linear filters used in the system will now be 
described. 

[0120] The frequency response plots are in decibels against Hz. All of the filters are symmetrical linear phase FIR 
filters, meaning that they delay all frequencies equally, so no phase response is shown. The time-domain plots are 

is impulse response waveforms. These show the output from the filter, for an input consisting of a single input sample of 
value 'V at t=0, all other input samples being zero. The time scale is in milliseconds for all plots. 
[0121] Fig. 7 shows the time domain response of the filter which is applied to the input signal before rate conversion 
(see step 20). It removes frequencies which cannot be represented at a sample rate of 11025. Fig. 8 shows the fre- 
quency response of the same filter. 

20 [01 22] The bandpass filter of step 24 is used in the beginning of the signal path 22 to extract the band of interest. Figs. 
9 and 10 show its frequency response and time domain response, respectively. 

[0123] The bandpass filter of step 80 is used in the Locator to remove the portion of the signal in the band of interest. 
Figs. 11 and 12 show its frequency response and time domain response, respectively. The length of the fitter, as a 
number of taps, depends on the sample rate. The plots are for 48000 Hz. The shape of the graphs will change only 
25 slightly for other sample rates. 

Ping Waveform 

[0124] Figs. 13 and 14 show a signal template for the ping waveform which is used in the insertion process. It is 
30 related to the filter response plots of Figs. 7-12 because band-pass filter design techniques are used to generate the 
ping waveform. Its band-limited spectral content is important to the invention because it must be within the bounds of 
the bandpass filter techniques described herein for creation and detection of the code. The Ping Waveform Spectral 
Content is shown in Fig. 13. The Ping Waveform Time Domain is shown in Fig. 14. 

35 A da pt ive M a sking 

[01 25] Another important feature will now be explained in connection with Fig. 1 . The data in Output File 1 3 generates 
audio sound, when reproduced through suitable equipment, which is used by step 15 of Fig. 1 to perform a subjective 
audio quality test based on psychoacoustic factors. Thus, testing process 15 could involve a group of human test sub- 
40 jects who listen to the audio and provide a descriptive feedback of audio quality, such as "I hear popping noises" or "I 
hear dropouts". The severity of each quality impairment is also ascertained on a scale of, say, 1 to 5 with 5 being the 
worst. 

[0126] Based on the derived feedback information of (1) the nature of the impairment and (2) its severity, a decision 
is made by step 1 7 regarding whether and how parameters 8 are to be modified. One embodiment of parameter control 
45 process 17 relies on a human operator who utilizes the feedback information and a set of empirically derived rules for 
modifying parameters 8. As an example of such rules, for popping noises assigned a value of 3 on the scale of 1 to 5, 
the PingGain is reduced by 10%. 

[0127] Another embodiment automates process 1 7 by developing a lookup table using the feedback information from 
step 17 as inputs. Step 17 then outputs control signals to suitably vary the parameters 8. 
so [01 28] The modified parameters 8 are used to embed the code as Locator 7 and Inserter 1 1 are performed once again 
to derive a new Output File 13. Test 15 and process 17 can continue to be applied iteratively for modifying parameters 
8 until an Output File 13 is derived which generates audio that satisfies the assigned audio quality requirements. 

DECODER 

55 

[0129] The decoder processes digital information derived from a received audio signal, including a waveform with an 
embedded code, typically it is picked up from an RF receiver. The analog signal is digitized to derive a Decoder Input 
File in WAVE format, comparable to Input File 1. 
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[0130] As shown in Fig. 15, the Encoded Audio Input File 118 is processed by step 120 which is analogous to step 
20 of Fig. 2. Step 124 of Fig. 16 is analogous to step 24 to provide a bandpass filter output. Operation 125 provides 
signal normalization, as explained below, and a normalized input signal is outputted by step 132. 
[0131] Ping identification is performed by step 134. which isolates candidate pings from the band-limited power-nor- 
malized signal. Each candidate ping is associated with a time point and an amplitude, both of which are determined via 
a template correlation process, as explained below. 

[01 32] In symbol identification step 1 36, the sequence of pings, identified in the first stage are examined to find pairs 
of pings, each pair of which forms a symbol. This is necessary because "false" pings occur with some regularity in the 
audio product source material itself. The decoder detects symbols by looking for pairs of pings with the correct relation- 
ship to each other. Once the symbols are identified, code assembly step 138 then reconstitutes the embedded code 
detected in the audio signal. 

[0133] Normalization operation 125 includes the various steps shown in Fig. 1 6. These are designed to match steps 
26-30 of Fig. 2 so that the signals outputted by the encoder and processed by the decoder are matched as well More 
specifically, step 126 is comparable to step 26. Step 128 is comparable to step 28. Step 129 is comparable to step 29 
15 However, step 130 differs from step 30 in that step 30 includes inversion, but step 130 does not. Therefore the signal 
from step 1 30 is the inverse of the output from step 30. This inversion by the decoder constitutes an amplitude normal- 
ization which removes variations due to gain or attenuation experienced during sending, playback and receivina ooer- 
ations for the audio product. ^ 
[0t34]— f^tails^f^tep5n34n36-aTm3gwill now be provided. 
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[01 35] Ping Identification step 1 34 is shown in detail by Fig. 1 7. It compares the normalized input signal derive* by 
step 132 with a ping waveform template, as per step 1 50 in Fig. 1 7. Details of this template have been provided above 
A circulating buffer stores as many samples of the normalized input signal as there are data samples in the ping wave- 
form template. Each sample of the input signal is compared with its respective template data sample. The RMS error 
for each sample comparison is caicuiaied to derive an error function E ( and an overall scaling factor K, per step 152 as 
25 described below. 

[01 36] The comparison algorithm finds the value of k, which minimizes the error function 



where tj is the fixed template and x i+j is the normalized input signal being analyzed. This can be expanded as 



40 This is minimized by setting the derivative with respect to Kj to zero, resulting in: 



J 



[01 37] Since tj is a fixed template, the denominator in the above is a constant which can be pre-calculated 
[0138] Calculation of 



J 



allows kj to be calculated. The polarity of K f determines the polarity of the received ping. Additionally, calculation of 
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5 

allows Ej to be found. 

[01 39] The decoder performs this comparison to search for pings on an 11 025 Hz signal. The search is actually done 
to a resolution of 44100 Hz. This is done by using four separate templates (representing 1/4-sample shifts of the ideal 
10 ping template relative to 1 1025 Hz). At each iteration, the template with the largest 



J 

15 



and, thus, the largest k and lowest E, is found. This provides a higher position resolution for locating pings, and a more 
accurate measurement of the ping's magnitude and closeness of match. 
20 [0140] Thus, the calculated value of Kj minimizes the error E, between the input signal and the ping templat^. That 
same value of K { is used to scale the magnitude of the ping template, and Kj is regarded as the ping magnitude. 
[0141 ] In order for a ping to be recognized as a candidate and passed to the next phase, the following two conditions 
must be met: 

25 * The error must be sufficiently small relative to the ping magnitude. The ping magnitude is divided by the square root 
of the error. This result is effectively a "shape matching" metric; scaling the input has no effect on this ratio. The 
ping is rejected, per step 154, if the ratio is less than parameter PingRatThresh (labeled "A M in Fig. 17 for conven- 
ience), which is dimensionless. A high ping magnitude value with a low error value constitutes a good match. 
• The magnitude of the ping must be at least MinPingMag (labeled "B" in Fig. 1 7 for convenience), as per step 1 56. 

30 This is in signal units. 

[0142] If the decisions in steps 154 and 1 56 are favorable, then step 1 58 outputs this portion of the input signal as a 
candidate ping for further processing by symbol identification step 136. 

[0143] Even though a portion of the input signal has been accepted as a candidate ping, it may in fact be an impostor. 

35 To aid in the determination of proper pings, the coding method uses a pairing of pings to create symbols. The symbols 
that are inserted by the encoder have very clear and unique characteristics that make detection of false pings easier. 
[0144] Details of symbol identification are shown in Fig. 18. Test 160 compares the time between two consecutive 
candidate pings against a threshold C which is less than the fixed time between consecutive pings (as explained 
below). If the spacing is greater than the threshold, step 161 outputs the signal as a real ping for further processing in 

40 connection with symbol identification, if the two pings occur too close to each other for them to be a symbol pair, a test 
162 is performed to determine which ping is to be rejected. The magnitude of the first ping is multiplied by the parameter 
"NewPingShadow" (called "D" in Fig. 18 for convenience), which is normally set to 0.1. If the amplitude of the second 
ping is less than the result, it is discarded as a false ping. If it is larger, then it is considered per step 164 to be the start 
of a new symbol frame to be outputted as a ping by step 161 . 

45 [0145] Pings that are inserted as symbols will have the same amplitude at the time of insertion, as explained above 
in connection with the encoder. If they do not have the same amplitude during decoding, step 166 determines which of 
two consecutive pings is larger. The ratio of the amplitude of the larger ping to the smaller is compared by step 168 to 
the "MaxPingSkew" parameter (labeled "E" in Fig. 1 7 for convenience). The result must not exceed the threshold set by 
this parameter; otherwise, the pings are considered not to be a symbol pair. 

so [0146] The spacing between ping pairs in a symbol is fixed at 4.354 msec. The spacing between consecutive pings 
is calculated and compared in step 170 to this fixed spacing. The absolute discrepancy between the calculated spacing 
and the fixed spacing must not exceed the "MaxPingSpacDev" parameter (labeled "F" in Fig. 18 for convenience). 
[0147] Symbol insertion creates a "quiet" portion between pings. The power of the band-limited, normalized, signal is 
measured by step 1 72 within a one T time window about a point halfway between the candidate pings. In a true symbol, 

55 there is very little signal energy there. The product of the magnitude of the two pings is divided by the measured power 
signal. The result, as determined by step 174, must exceed the "MaxGap Ratio" parameter (labeled "G" in Fig. 18 for 
convenience) for the two pings to be considered a true symbol. If so, step 176 outputs the pings as a true symbol. 
[0148] The true symbols outputted by step 136 are then collected and the resultant code is determined by code 
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assembly step 138 by applying the information set forth above in Table 7 

[0149] Representative values of the decoder parameters are provided in the following table. 



Table 8 



10 



75 



Parameter 


Value 


PingRatThresh 


10 


MinPingMag 


200 


Threshold C 


3T-181 usee. 


NewPingShadow 


0.1 


MaxPingSkew 


5.0 


MaxPingSpacDev 


181 usee. 


MaxGapRatio 


250 
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Claims v 

1. A method of embedding a digital code in a digitized audio product, comprising the steps of: 
filtering the digitized audio product to a frequency band of interest- 

The method of claim 1 . wherein all of said plurality of segments have a uniform duration. 

mJL" 1 ??* 1 °1 Claim 1 ' fUrther com P risin 9 *e step of determining the total power for at least said narticutar 
ment. and perform.ng said inserting step only « said total power is above a J«««5*2J5l!iSi^^ 9 " 

A method of embedding a digitized code in a digitized audio product, comprising the steps of: 

filtering the digitized audio product to a frequency band of interest- 

deriving sa.d digitized code from said coding signal; and 
embedding said digitized code into said audio product. 
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6. The method of claim 5. wherein said frequency band of interest is approximately 0 to 5000 Hz, and said coding sig- 
nal has energy spread across approximately 1500 to 4000 Hz. 

7. The method of claim 5, wherein said digitized code is derived based on polarity of said coding signal. 

5 

8. The method of claim 5, wherein each bit of said digitized code is derived based on a plurality of said coding signals. 

9. The method of claim 5, wherein said coding signal is derived from an ideal mathematical impulse function to which 
a steep bandpass filter is applied. 

10 

10. A method of providing a digitized code to be embedded in a digitized audio product, comprising the steps of: 

providing said digitized code as a series of binary bits; 
dividing said binary bits into groups, each group having a plurality of bits: 
75 providing coding signals to represent said bits, respectively: 

deriving a symbol from said coding signals for each of said groups, each symbol having a plurality of said cod- 
ing signals with a preset spacing therebetween. 

11. The method of claim 10, wherein each symbol has coding signals equal in number to bits in a group to which such 
20 symbol corresponds. ^ 

12. The method of claim 10, wherein the coding signals are identical to each other in shape, with one binary bit corre- 
sponding to a coding signal of one polarity, and the other binary bit corresponding to a signal of the other polarity. 

25 13. The method of claim 10. wherein a symbol consists of two of said coding signals. 

14. The method of claim 10, wherein a symbol consists of three of said coding signals with equal spacing between 
adjacent ones. 

30 15. A method of encoding and decoding a digitized code embedded in a digitized audio product, comprising the steps 
of: V 

deriving said digitized code in a form of start, data and end symbol types, each symbol representing a plurality 
of bits, and each bit being associated with a coding signal of given polarity; 
35 generating said start type of symbol to consist of a plurality of said coding signals all of which have the same 

designated polarity; 

embedding said digitized code in said digitized audio product; 
detecting said digitized code embedded in said audio product; and 

decoding said detected digitized code by determining whether the polarity of the coding signals on said start ' 
40 type of symbol is said designated polarity and, if not, inverting the polarity of said coding signals in said data 

and end types of symbols. 

16. A method of embedding a digitized code in a digitized audio product, comprising the steps of: 

45 identifying segments of the digitized audio product into which the digitized code can be embedded based on 

predetermined criteria; 

generating portions of the digitized code for insertion into said segments, respectively; 
removing the digitized audio product within said identified segments except for a predetermined small percent- 
age of amplitude to generate modified segments; and 
so inserting said portions of the digitized code into said modified segments, respectively. 

17. A method of embedding a digitized code in a digitized audio product, comprising the steps of: 

analyzing the digitized audio product to derive measured values for designated characteristics thereof; 
55 locating segments of the digitized audio product, based on said derived measured values and a set of prese- 

lected parameters, into which the digitized code can be inserted so as to be masked; 
inserting the digitized code into said located segments; 

determining whether a degree of masking of the inserted digitized code meets a predetermined level and. if 
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not, modifying values of at least one of said set of preselected parameters, and then performing said locatina 
and inserting steps again with said modified values. 

18. A method of embedding a digitized code in an audio product, comprising the steps of: 

dividing said digitized code into preselected portions; 
representing said portions by a plurality of coding symbols, respectively; 

determining spacing of said coding symbols from each other to be used for embedding the digitized code within 
the audio product so that said spacing is greater than a predetermined minimum, and 
inserting the coding symbols within the audio product based on said determined spacing. 

1 9. The method of claim 1 8, wherein said predetermined minimum spacing for a present symbol is in relation to a loca- 
tion of a previous symbol, 

is 20. The method of claim 18, wherein said predetermined minimum spacing for a present symbol is in relation to a loca- 
tion of the immediately preceding previous two symbols. 

21 . The method of claim 18, wherein said spacing is derived to produce a random spacing. 



w 



20 22. A method of decoding an audio product into which a code of digitized coding signals has been embedded com- 
prising the steps of: ' w 

obtaining a digitized audio product; 

... „ t „ M auulv ^ivaju^i wan a temple ot a cooing signal to identify candidate coding signals 

25 based on shape; M y 

comparing pairs of sequential candidate coding signals with each other based on preselected characteristics 

to identify which ones constitute the coding signals; and 

reconstructing said cede from said coding signals identified by said comparing step. 

30 23, The method of claim 22, wherein said comparing step compares amplitude. 

V 

24. The method of claim 22, wherein said comparing step compares a predetermined spacing with spacing between a 
sequential pair of said candidate coding signals. 

35 25. Apparatus for embedding signals representing a digitized code in an audio product, comprising: 

a bandpass filter receiving the audio product and having an output in a frequency band of interest' 
a signal generator outputting a coding signal derived from a band-limited impulse function with a waveform 
having its energy confined to and evenly spread across at least a portion of the frequency band of interest- 
means for deriving the signals for representing said digitized code from said coding signal; and 
means for embedding said signals representing the digitized code into said audio product.' 

26. The apparatus of claim 25, wherein said signals representing the digitized code are derived based on polarity of 
said coding signal. 3 

27. The apparatus of claim 25. wherein each bit of said digitized code is represented based on a plurality of said codina 
signals. a 

28. The apparatus of claim 25, wherein said coding signal is derived from an ideal mathematical impulse function to 
so which a steep bandpass filter is applied. 

29. A method of providing a digitized code to be embedded in an audio product, comprising the steps of: 



40 



45 



55 



representing said digitized code by a plurality of analog symbols to be embedded in the audio product- 
obtaining a plurality of data samples corresponding to a digitized audio product; 

filtering a portion of the plurality of data samples of the digitized audio product to a frequency band of interest- 
calculating an RMS envelope signal for said filtered portion of the digitized audio product; and 
controlling an amplitude level of said symbols in accordance with said calculated RMS envelope signal 
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30. A method of decoding an audio product into which a code of coding signals has been embedded, comprising the 
steps of: 

obtaining a digitized audio product with the code of coding signals being embedded therein; 
storing a digitized template of a coding signal as a given number of data samples; 

storing from one portion of the digitized audio product a number of data samples equal to said given number 
of data samples of said stored template; 

comparing corresponding data samples of the stored template and audio product with each other to determine 
an error function between said template and said stored portion of the digitized audio product; 
determining a scaling factor for the stored audio product which produces a minimum error from said error func- 
tion; 

determining amplitude for said one portion of the audio product from said scaling factor; and 

combining said minimum error with said amplitude to recognize whether a coding signal is included in said 

stored portion of the digitized audio product. 

31. The method of claim 30, wherein said step of combining comprises dividing the error by the amplitude to produce 
a result, and said coding signal is recognized if said result exceeds a given threshold. 

32. Apparatus for decoding an audio product into which a code of coding signals has been embedded, comprising: 

means for obtaining a digitized audio product with the code of coding signals being embedded therein; 
means for storing a digitized template of a coding signal as a given number of data samples; 
means for storing from one portion of the digitized audio product a number of data samples equal to said given 
number of data samples of said stored template; 

means for comparing corresponding data samples of the stored template and audio product with each other to 
determine an error function between said template and said stored portion of the digitized audio product; 
means for determining a scaling factor for the stored audio product which produces a minimum error from said 
error function; 

means for determining amplitude of the audio product from said scaling factor; and 

means for combining said minimum error with said amplitude to recognize whether a coding signal is included 

in said stored portion of the digitized audio product. V 

33. The apparatus of claim 32, wherein said combining means comprises means for dividing the error by the amplitude 
to produce a result, wherein said coding signal is recognized if said result exceeds a given threshold. 

34. Apparatus for providing a signal representing a digitized code to be embedded in an audio product, comprising: 

means for providing said digitized code as a series of binary bits; 

means for dividing said binary bits into groups, each group having a plurality of bits; 

means for providing coding signals to represent said bits, respectively; and 

means for deriving a symbol from said coding signals for each of said groups, each symbol having a plurality 
of said coding signals with a preset spacing therebetween. 

35. The apparatus of claim 34, wherein each symbol has coding signals equal in number to bits in a group to which 
such symbol corresponds. 

36. The apparatus of claim 34, wherein the coding signals are identical to each other in shape, with one binary bit cor- 
responding to a coding signal of one polarity, and the other binary bit corresponding to a signal of the other polarity. 

37. The apparatus of claim 34, wherein a symbol consists of two of said coding signals. 

38. The apparatus of claim 34, wherein a symbol consists of three of said coding signals with equal spacing between 
adjacent ones. 

39. Apparatus for decoding an audio product into which coding signals representing a digitized code have been 
embedded, comprising: 

means for obtaining a digitized audio product with signals representing said digitized code being embedded 
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therein; 

c'itss zszex** ■* *** * a ,mpia,e 01 a s ** * "«* — . 

second means for comparing pairs of sequential candidate coding signals with each h«««H „ 
lected characteristics to identify which ones constituted the o«nf££Z£ °" P ' eS6 - 

means for reconstructing said code from said coding signals identified by said second comparing step. 

40. The apparatus of claim 39, wherein said second comparing means compares amplitude. 

' ^^^^^^^^^^^^ 

42. Apparatus for embedding a digitized code in an audio product, comprising: 

Z7^a!ZT eniS * ^ 3Udi0 int ° WhiCh the di9iti26d COdS «" be ^sed on 

ZE ZSSS° Si9na,S r6PreSentin9 ° f *• di9ifeed COde for «*> said seg- 

^IZ^^^^f f «* » » determined sma„ per- " 

mZX££ COdin9 Si9 " a,S rePr6Sentin9 P ° rti0nS * ,he cod ° ^ - modified sV 

43. Apparatus tor embedding a digital code in an audio product, comprising: 

means for filtering the audio product to a frequency band of interest- 
means for determining a tonality indication for each of a plurality of seoments of the f.iwoH * - 

means for representing said digital code with symbols- and 

.f sa.d tonalrty in d,cat.on. .nd,cates a relatively uniform power distribution in said particular ^ST^ 

44. Apparatus for embedding a digitized code in a digitized audio product, comprising: 

means for analyzing the audio product to derive measured values for designated characteristics 

means for locating segments of the audio product, based on said derived imSSS!^^^ 

SsSg'^^ 

locating and inserting steps again ba Sed 0 n said moSw I vies P ^ and Perf ° rm,n9 said 

45. Apparatus for encoding and decoding a digitized code embedded in an audio product, comprising: 

means for representing said digitized code by symbols in a form of start data and end *™hr* 1 , 

sartsymbolsrep^ 

.r*m^^ 

means for embedding the symbols representing said digitized code in said audio product- 
means for detecting said symbols embedded in said audio product- 
means for determining whether the polarity of the coding signals for said detected start tvn* «f ..^ • 

means for decoding said digitized code based on the coding signals produced by said determining 
46. A method of providing a digitized code to be embedded in an audio product, comprising the steps of: 



means. 
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representing said digitized code by a plurality of analog symbols to be embedded in the audio product; 
applying a frequency transform to said audio product to obtain a transform signal within a frequency band of 
interest; 

calculating power of the transform signal in each of a plurality of frequency groups, wherein each of said fre- 
5 quency groups includes a portion of said band of interest; and 

controlling an amplitude level of said symbols in accordance with said calculated power. 

47. The method of claim 46, wherein said amplitude level of the symbols is the RMS level. 

10 48. The method of claim 46, further comprising the step of obtaining a total power from the powers calculated for each 
of said frequency groups, and wherein said controlling step utilizes said total power. 

49. A storage medium storing processor implementable instructions for controlling a processor to carry out the method 
of any one of claims 1 to 24, 29 to 31 , and 46 to 48. 



15 



50. A storage medium storing an audio signal having a digital code embedded therein by the method of any one of 
claims 1 to 9 or 16 to 21. 



20 



51 . An audio signal having a digital code embedded therein by the method of any one of claims 1 to 9 or 16 to 21 . 

52. An electrical signal carrying computer implementable instructions for controlling a processor to carry out the 
method of any one of claims 1 to 24, 29 to 31 and 46 to 48. 
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